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About  Me 


•  Career: 

-  Present:  Team  Lead  for  the  CFD/Signature  Modeling, TARDEC 

-  2003-2008  Survivability  Specialist,  General  Dynamics  Land  Systems 

-  1999-2003  Research  Engineer,  ThermoAnalytics,  Inc. 

-  1994-1996  Test  Engineer/  Lab  Tech,  Whirlpool  Corporation 

•  Education: 

-  2002  MSME  Michigan  Tech 

•  Project:  Infrared  Signature  Modeling  of  the  AH-64  Apache 

-  1999  BSME  Michigan  Tech 

•  Classes  Taken  Specific  to  This  Thesis: 

-  Data  Mining 

-  Neural  Networks 

-  Intelligent  Systems 
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Basic  Problem 


Typical  Military  Vehicle  HVAC  Simulation 

•  CFD  models  of  vehicle  interior 
are  ~15  million  cells 

•  Solve  steady  state  HVAC  with 
64  cores  overnight 

•  Need  to  perform  heat  soaked 
vehicle  simulation  and  HVAC 
startup 

-  Would  take  at  least  a  week 

•  Particularly  interested  in 
equipment 

-  Want  operational  temps  in  less  than 
15  minutes 


Typical  Military  Vehicle  Equipment  Rack 
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Research  Objectives 


1.  Demonstrate  process  of  clustering  a  steady-state  CFD 
domain  into  a  transient  lumped  fluid  network  simulation 
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Research  Objectives 


1.  Demonstrate  process  of  clustering  a  steady-state  CFD 
domain  into  a  transient  lumped  fluid  network  simulation 

2.  Show  value  of  remapping  CFD  fluid  film  convection  to 
localized  fluid  nodes 


Which  H?  Remapping  Equation 
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4.  Show  new  ways  of  flowfield  visualization  using  clustering 
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Research  Objectives 


1.  Demonstrate  process  of  clustering  a  steady-state  CFD 
domain  into  a  transient  lumped  fluid  network  simulation 

2.  Show  value  of  remapping  CFD  fluid  film  convection  to 
localized  fluid  nodes 

3.  Investigate  the  use  of  clustering  to  track  temperatures 
around  specific  equipment  or  locations 

4.  Show  new  ways  of  flowfield  visualization  using  clustering 

5.  Validate  subvoluming  and  new  clustering  techniques 
against  fully  transient  CFD  data 
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State  of  the  Art:  Subvoluming 

Approach 


Simple  idea:  cut  planes  through  the  domain  and  track 
volume  fluxes  across  surfaces  to  make  control  volume 
network. 

Apply  a  fixed  1 5  w/m2  convection  coefficient  that 

convects  to  closest  node  technology  driven,  warfighter  focused. 
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Subvolumed  Model  /  Problems 


Subvolume  nodal  arrangement 
72  Nodes  Evenly  Spaced 


Y 


32,4  33,8  35,1 


36,4  37,8  39,1 


40,4 


Equipment  Rack 


l 


•  Volumes  cut  complete  across 
the  shelves. 

•  Multiple  equipment  associated 
with  each  node 
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Another  Problem  with 
Subvoluming 


Jagged  Plane  Edges 


Net  Into  Duct  =  550  CFM 


(CFD  Boundary  Condition) 


Net  Into  Subvolume  =  625  CFM 

(From  Subvoluming  macro) 
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Agenda 
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Data  Mining 


•  Data  mining  -  knowledge  discovery  in  large  sets  of  data 

•  Relatively  new  field  in  the  area  of  computer  science 

•  Uses/  Examples: 

-  Grouping  similar  documents  together 

-  Determining  consumer  spending  patterns  (market  segmentation) 

-  Risk  analysis  and  management 

-  Fraud  detection  and  detection  of  unusual  patterns  (outliers) 

-  DNA  and  bio-data  analysis 


Knowledge  Mining 


Knowledge  Discovery 
in  Databases 


Database  Mining 
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Data  Mining  -  Clustering 


•  Clustering  is  type  of  data  mining 

•  Unsupervised  classification  of  patterns  observations 
data  items  or  feature  vectors  into  groups  or  clusters  - 
Jain  [9] 

•  Clusters  should  be  externally  isolated  and  internally 
cohesive,  implying  homogeneity  within  clusters  and 
heterogeneity  between  clusters  -  Cormack  [10] 

•  Examples  of  use 

-  pattern  recognition 

-  image  processing 

-  market  research 

-  document  classification  for  web  searching 
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Factors  Involving  Data  Mining 


•  Selection  of  features  of  interest 

-  Quantitative  features: 

•  Continuous  values  (e.g.  weight). 

•  Discrete  values  (e.g.  number  of  computers) 

•  Interval  values  (e.g.  the  duration  of  an  event) 

-  Qualitative  features: 

•  Nominal  or  unordered  (e.g.  color) 

•  Ordinal  (e.g.  qualitative  evaluations  of  temperature  "cool"  or  "hot") 

•  Preprocessing  method 

-  Z-normalization 

-  Outlier  rejection 

•  Selection  of  objective  (goal)  function 

-  Maximize  separation 

-  Minimize  within  cluster  similarity 


TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED. 


UNCLASSIFIED 


UNCLASSIFIED 


Distance  Metrics 


Distances  are  normally  used  to  measure  similarity  or 
dissimilarity  between  two  data  objects 

•  Equation  1:  Euclidean  Distance 


f  d 


de  = 


\ 


Zk*  ~xj,k)2 

V  k= 1  J 


Equation  2:  Minkowski  Distance 


dm  =  -Xjjc)1 


X. 


k  =  \ 


Equation  3:  Cosine  Similarity  (dot  product) 

f  d  \ 


d  = 


V  k=l  J 


Equation  4:  Manhattan  Distance 

d 

dm=TJ(Xi,k  ~  Xj,k  ) 


k- 1 
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K-Means  Basic  Algorithm 


Objective  is  to  minimize  distance  to  cluster  centers  based  on  a 
Euclidean  distance. 


Pick  Random  Starting  Points 


Assign  Data  to  Closest  Distance  Centroid  Compute  a  New  Mean  Centroid 
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K-Means  With  no  Feature 
Weighting  (Only  x,y,z) 


Purely  based  on  x,y,z  as  features,  k-means  produces  equal 
"inertia"  clusters  about  each  centroid 
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K-Means  With  Weighting  and 
Multiple  Variables 


•  Produces  equal  "inertia"  clusters  about  the  weighted 
multivariate  feature  dimensions. 

•  Weight  x,y,z  distance  much  higher  to  keep  clusters 
compact. 
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cfdMine  GUI 
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cfdMine  Design 


•  Commercial-quality  C++  tool 

-  QT  for  user  interface 

-  Eigen  stl  library  for  Matrices 

-  Kd-tree  for  matching  elements 

-  Modified  dated  Ensight  reader  for  polyhedral  mesh 

-  5000+  lines  of  code 


K-Means  Implemented  with  Euclidean  Distance 
Weighted  features  of  interest 


-  Temperature 

-  X,Y,Z  distances 

-  Velocity  magnitude 

-  Velocity  direction  (as  dot  product) 

Preprocessing  method 

-  Z-normalization  used 


Ensight  Gold  .geo  BINARY:  CaselOOOOO 

Region  1  v 


Load  Ensight 


#  Clusters 


#  Retries  ConvergenceCrit  Relaxation  Maxlter 

15  jl  0.0100  Jj  1.00  jl  300  JJ 

Temp  Wt  Geom  Dist  Wt  Veloc  Dir  Wt  Veloc  Mag  Wt 

1 0.20  1 0.70  0  0.05  ~Z\  [°05  C] 


Specify  Fixed  Clusters 
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Basic  cfdMine  Process 


Note:  The  nodal  network  is  set  up  in  MuSES  using  a  thermal 
link  from  every  element  to  each  cluster  fluid  node.  (A  first, 
but  not  ideal) 
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Which  Knob  Does  What 
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Visualizing  Clusters  -  Case  1 
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Generic 
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Clusters  - 
cle  Interior 
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Importance  of  Volume  Weighting 


K-Means  Volume  Weighted  Iterative  Function 


v 


X 


Tii 


1 

xeSi 


x  ■  vx 
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Reason  To  Weight  -  Typical 

Meshes 
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Fixing  Clusters  Around 
Equipment 


Clustering  About  the  Centroid 


Clustering  About  the  Bounding  Box 


max 


X 


% bbox_min  ~  ^  xmin)  ?  xmin  :  ^  ^max )  ? 

ybboxjnin  ~  (}7  ^  7mm)  ?  7mm  :  (y  ^  Yin  ax )  ?  /max  :  Y 
^bboxjnin  ~  ^  )  ?  ^min  '  (%  ^  :  ^ 


ti  IS  tr 


—  ^0^  ^Wwx_mm  )  ^O7  yz?Z?ox_mm  )  ^frfrox_mm  ) 


Clustering  around  the  bounding 
box  is  quick  and  efficient. 


UNCLASSIFIED 


UNCLASSIFIED 


Cluster  Size  Around  Equipment 


Ensight  Gold  .geo  BINARY: 

7V\  Region  1 


CaselOOOOO 


E^i 


ConvergenceCrit 


C  *> 


Geom  Dist  Wt  Veloc  Dir  Wt 


Larger  weights  allow  the  fixed  location  equipment- 
clusters  to  be  smaller 
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Fixing  Clusters  Around 
Inflow  &  Outflows 


Important  to  Place  Clusters  at 


•  Required  to  enforce  flow  network  conservation  of  volume  flow 

•  Usually  an  area  of  interest 
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Bifurcation  Problem  at 
Inflow/ Outflow 


•  Usually  better  to  use  centroid  distance  (as 
opposed  to  bounding  box)  from  inflow/  outflow 

•  Added  feature  to  specify  cluster  temperature, 
velocity  at  fixed  nodes,  to  further  avoid  this 
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Validating  Flux  at  The  Interface 

Boundaries 
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This  case  is  fairly  idealized:  Planes  are  perpendicular  to  flow,  fine 
mesh,  and  no  major  temperature  gradients. 
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Agglomeration  -  Dealing  With 

Islands 
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Convergence  Plots 
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Typical  k-means  convergence  plot  for  50  “zoomed-in”  plot  of  50  random-start  k- 
random  start  trials  means  trials 
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cfdMine  Mahalanobis 


File  About 
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Mahalanobis 


15 


From  MuSES 


From  CFD 


K-Means 

Mahalanobis 


Label 

X 

Y 

Z 

BBox 

TempWt 

GeoWt 

VDirWt 

VMag’ 

1 

Duct 

0.019268 

4.82436 

2.68189 

V 

1 

5 

1 

1 

2 

C_Reiay_Box 

-0.523846 

5.33141 

2.47283 

Y 

1 

5 

1 

i 

3 

C_RPCU 

-0.547708 

5.20134 

2.1773 

1 

5 

1 

1 

4 

C DCE 1 

-0.526248 

5.02561 

2.82669 

>/ 

1 

5 

1 

1 

< 


<  > 


Calculation  Status 


Agglom 


Calc  Flux 


Reading  Ensight  God  geometry  hie... 
Reading  element  results  data... 
Temperatures 
Centroids 
Velocities 
Vb  lurries 

Populating  data  structures... 


Write  out  Ensight  file  for  viewing 
Setup  in  MuSES  TDF  file 


Write  Ensight 

FINISHED 

Setup  MuSES 

A 
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Mahalanobis  Distance  Metric 


12 


2 


Euclidean  Distance 
(Standard  K-Means) 


Mahalanobis  Distance 


Euclidean  distances  fit  clusters  to  hyperspheres. 
Mahalanobis  fits  data  to  an  oriented  hyperellipsoid. 
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Mahalanobis  Distance 


a2=  o i  -  i uky  2  1(xi-  nk) 


where 


"°ii 

°12 

3 

tr* 

Z  = 

^21 

°22 

°2L 

.aLl 

°L2 

oLL. 

covariance  matrix 


Note:  An  identity  matrix  results  in  Euclidean 
distance. 


Mahalanobis  cannot  be  used  directly  with  K-means.  Adds  36 
unknowns  and  drops  into  non-optimal  local  minima  even  faster. 
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Example:  Univariate 
Mahalanobis  Distances 


Euclidean  distances  are  one  in  both  cases,  but  Mahalanobis 
distance  of  A  =1  and  2 
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cfdMine  Mahalanobis  /  New 

Algorithm 


File  About 


K-Means 

Mahalanobis 


Agglom 


Calc  Flux 


Calculation  Status 


Reading  Ensight  Gold  geometry  file... 
Reading  element  results  data... 
Temperatures 
Centroids 
Velocities 
Vblumes 

Populating  data  structures. . . 


Write  out  Ensight  file  for  viewing  Write  Ensight 

Setup  in  MuSES  TDF  file  5etuP  MuSES 


FINISHED 


Initial  "m"  pure  Euclidean 
iterations 


Blend  inv  covariance  w/ 
Euclidean(identity) 


Only  update  covariance 
"n"  iterations 


TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED. 


UNCLASSIFIED 


UNCLASSIFIED 

Results  -  Better  Data  Separation 


Mahalanobis  clusters  Pure  Euclidean 


15  clusters,  75%  Euclidean  and  15%  Mahalanobis 


15  clusters,  100%  Euclidean 


Mahalanobis  resolves  entire  "jet"  in  Case  1,  inlet  to  outlet. 


UNCLASSIFIED 


Agenda 


•  About  Rob  E  Smith 

•  Research  Objectives 

•  Quick  Introduction  to  Data  Mining 

•  cfdMine  Tool 

•  cfdMine  Mahalanobis 

•  Validation  Results 

•  Conclusions/  Next  Steps 
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Validation  Goal/  Method 


•  Subvolumed  transient  MuSES  nodal  network 

-  Based  on  steady-state  CFD 

-  Transient  30  minute  run 

•  Clustered  transient  MuSES  nodal  network 


Based  on  steady-state  CFD 
Transient  30  minute  run 


•  Compare  to  fully  transient  CFD  for  30  minutes 

-  Extract  average  cell  temperatures  from  clusters  each  minute 


TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED. 


UNCLASSIFIED 


UNCLASSIFIED 


Study 
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Equipment  Rack  Close-Up 


Duct:  550  CFM,  9° 
Temperature  Drop 


DCE1 


4  Outlets 


Relay  2 


Router  3 


DCE3 


Router 

Relay 
Router  4 

■  RPCU 
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Subvolumed  Nodal  Arrangement  _.A 

a  S\  ( 


Subvolume  nodal  arrangement 
72  Nodes  Evenly  Spaced 
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Clustering  Nodal  Arrangement 


•  100%  Euclidean 

•  Minimal  time  spent  on  clustering  (not  tailored) 

•  35  Nodes 

-  10  around  equipment 

-  5  around  inlet/  outlets 
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Validation  Clustering  View  2 
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Validation  Clustering  View  3 
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Clustered  Network  as  Viewed 

in  MuSES 


File  Edit  View  Tools  Units  Window  Help 
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1  3  5  7  9  11  13  15  17  19  21  23  25  27  29 


Time  (Minutes) 


Router  2 
Fluid 


CFD 

DCE1,  Router3  Subvolumed 
Clustered 
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1  3  5  7  9  11  13  15  17  19  21  23  25  27  29 

Time  (Minutes) 


Relaybox 

Fluid 


CFD 

Relay  BoxSubvolumed 
Clustered 
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1  3  5  7  9  11  13  15  17  19  21  23  25  27  29 

Time  (Minutes) 


RPCU  Fluid 


CFD 

RPCU  Subvolumed 
Clustered 
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Demonstrated  Need  For 
Localized  Convection 


Clustering  technique  validation  surface  temperatures  on  the 
hull  exterior  at  the  end  of  the  30  minute  simulation 


S1I _ 615 _ 62* _ K5 _ Ml _ 635 _ 64  0 _ 845 _ 85* 

M  — IL 


z 


Y 


Subvoluming  technique  validation  surface  temperatures  on  the 
hull  exterior  at  the  end  of  the  30  minute  simulation 


81* _ 615 _ 82* _ 625 _ 63* _ «5 _ 84* _ 645 _ 65* 


Clustering  technique  validation  surface  temperatures  on  the  equipment  Subvoluming  technique  validation  surface  temperatures  on  the  equipment 
rack  at  the  end  of  the  30  minute  simulation.  rack  at  the  end  of  the  30  minute  simulation. 
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Remapping  Problem 


Sometimes: 


i  _  hfiimiTfilm  Twall) 

ft local  ~  T  _T 

1  local  1  wall 


produces  a  negative  h.local 

Options: 

•  Set  to  zero 

•  Use  absolute  value  < -Recommended 


Validation  Case  Elements  Which  Have 
Negative  Convection  When  Remapped 


TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED. 


UNCLASSIFIED 


UNCLASSIFIED 


Conservation  of  Volume  Problem 


•  It  was  noted  the  network  is  leaky  for  both  subvoluming 
and  cluster  based  networks. 

-  Due  to  tolerances 

-  Up  to  10%  imbalance  seen  on  test  case 

-  Also  should  have  been  based  on  conservation  of  mass. 

•  Volume  Balance  Fix: 

•  Start  with  the  first  cluster  and  assess  all  incoming  and  outgoing 
advection  links.  If  there  is  a  source  outside  the  domain,  enforce  this 
in  the  calculation  as  well. 

•  Divide  the  imbalance  in  half  and  apply  half  to  the  incoming  and  half  to 
the  outgoing  flow  to  make  them  balance  to  zero  (or  reduce  by  a 
relaxation  factor).  Do  this  proportional  to  each  advection  link  divided 
by  the  total  incoming  or  outgoing  flow. 

•  Evaluate  each  cluster  according  to  step  2  successively  until  each 
cluster  exhibits  no  further  imbalance. 
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Density  Plot 
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Agenda 


•  About  Rob  E  Smith 

•  Research  Objectives 

•  Quick  Introduction  to  Data  Mining 

•  cfdMine  Tool 

•  cfdMine  Mahalanobis 

•  Validation  Results 

•  Conclusions/  Next  Steps 
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Conclusions 


•  The  new  overall  process  of  clustering  a  steady-state 
CFD  domain  into  a  transient  lumped  fluid  network 
simulation  performs  well 

•  It  is  absolutely  critical  to  conserve  mass  within  the  fluid 
network 

•  Remapping  CFD  fluid  film  convection  to  localized  fluid 
nodes  is  critical  to  an  accurate  solution 

•  Mahalanobis  distance  is  better  than  Euclidean,  but  too 
computationally  intensive  for  the  present  constraints 

•  Clustering  is  an  effective  method  to  track  temperatures 
around  specific  equipment  or  locations 

•  Clustering  provides  a  beneficial  new  way  to  visualize 
flowfields 

•  Subvoluming  should  be  used  with  extreme  caution 
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New  Contributions  to  the  Field 


•  First  to  implement  clustering  to  simplify  a  CFD  network  into 
a  simplified  network 

•  First  known  volume  weighted  clustering  of  a  CFD  domain. 

•  First  known  remapping  of  convection  coefficients  were 
remapped  to  localized  nodes  in  a  network 

•  First  use  of  fixed  nodes  where  the  nodes  were  fixed  at  areas 
of  interest  during  clustering  for  the  purpose  of  a  simplified 
nodal  network. 

•  Proposed  new  methods  to  stabilize  the  Mahalanobis  distance 
and  make  it  usable. 

•  Showed  several  new  ways  to  visualize  flow  data 
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Future  Work 


•  Implement  conservation  of  mass 

•  Investigate  use  of  unidirectional  flux  linkages 

•  Look  into  enhanced  subvoluming 

-  Control  volumes  around  equipment 

-  Localized  convection 

•  Investigate  importance  of  transient  versus  steady-state 
convection 

•  Convection  "view  factors" 

-  Solve  a  steady-state  CFD  solution 

-  Release  particles  from  all  wall  elements  and  boundary  elements 
(perform  a  particle  trace) 

-  Account  for  the  mixing  and  the  fraction  of  volume  of  influence 
each  wall-released  particle 

-  Ultimately  track  the  influence  each  wall  element  /  boundary  has 
on  each  other  element/  boundary 


TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED. 


UNCLASSIFIED 


TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED. 


BACKUP  SLIDES 
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50  Nodes 
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50  Nodes 
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K-Means 


•  K-Means  was  first  suggested  in  1962  by  Sebestyen  [15] 
and  independently  in  1967  by  MacQueen  [16].  A 
detailed  history  and  full  pedantic  discussion  of  various 
deviations  may  be  found  in  "K-Means  Clustering:  A  Half- 
Century  Synthesis"  [17]. 

•  Performance  function:  Minimize 

2 

-  Sum  Squared  Error  —  £[Li  | x  —  m/  \  \ 

•  K-Means  Iterative  Function 
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K-Means  Basic  Algorithm 


•  for  i=l  to  number  of  tries 

-  repeat 

•  randomly  select  cluster  nodes 

•  compute  distance  of  every  point  in  domain  to  nodes 

•  assign  points  to  cluster  based  on  minimum  distance 

•  compute  volume  weighted  means  to  find  new  nodes 

•  track  the  solution  with  minimum  total  error 

-  until  no  change 

•  end  for 
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point 


two  node  bar 


2 


three  node  bar 


six  node  pentahedron  (wedge)  fifteen  node  pentahedron  (wedge) 
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Example  Clustering 
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Number  of  Clusters 


10  Clusters 


Result:  10  clusters  is  the  minimum  for 
this  particular  case  to  capture  steam. 


15  Clusters 
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of  Test  Case  1 
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K-Means  Mahalanobis 
Convergence 


Typical  k-means  convergence  plot 
for  33  random  start  trials  on  a 
vehicle  duct  geometry.  The  lines 
are  translucent.  The  black  lines 
show  25  runs  with  a  50% 
Mahalanobis/Euclidean  blend 
versus  the  8  red  lines  which  show 
75%  Mahalanobis.  Note  the  first  5 
iterations  are  pure  Euclidean 


“Zoomed-in”  plot  of  typical  k-means 
convergence  plot  for  33  random  start  trials 
on  an  vehicle  duct  geometry.  The  lines  are 
translucent.  The  black  lines  show  25  runs 
with  a  50%  Mahalanobis/Euclidean  blend 
versus  the  8  red  lines  which  show  75% 
Mahalanobis.  Note  the  first  5  iterations  are 
pure  Euclidean 
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Data  Mining  /  Clustering 


•  Primary  two  categories  for  data  clustering  are: 

1.  Hierarchical 

2.  Partitional. 

•  Hierarchical  clustering  is  based  on  the  notion  that 
objects  are  more  related  to  nearby  objects  than  to 
objects  farther  away. 

-  Hierarchical  clustering  can  be  computed  bottom-up  or  top-down. 
Hierarchical  agglomerative  clustering  is  starting  with  single 
elements  and  aggregating  them  into  clusters. 

-  Hierarchical  divisive  clustering  is  starting  with  the  complete  data 
set  and  dividing  it  into  partitions. 

•  Partitional  clustering  operates  on  data  globally 

-  Exemplified  by  k-means  where  given  an  initial  set  of  "k"  means, 
points  are  assigned  to  the  most  similar  "k"  cluster. 

-  Centroid-based  clustering  optimization  is  ultimately  considered  an 
np-hard  problem,  but  is  possible  to  solve  for  most  practical  cases. 
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New  cfdMine  Mahalanobis 
Algorithm 


Mahalanobis  cannot  be  used  directly  with  K-means.  Adds  36 
unknowns  and  drops  into  non-optimal  local  minima  even  faster. 


1.  Pick  random  cluster  centers  from  the  domain  and  assign  elements  randomly 

2.  Z-normalize  all  the  data  in  the  domain 

3.  First  m  iterations  are  Euclidean 

4.  If  iterations  >  n,  compute  the  covariance  matrix,  otherwise  identity  matrix 

5.  Divide  the  x,y,z  3x3  inverse  covariance  matrix  by  the  maximum  value  and 
multiply  by  the  weight  of  importance  for  physical  distance.  For  temperature, 
velocity  magnitude,  and  velocity  direction,  divide  the  associated  rows  and 
columns  by  the  diagonal  value. 

6.  Blend  the  inverse  covariance  matrix  with  an  identity  matrix  with  variable 
weights  on  the  diagonal. 

7.  Compute  the  Mahalanobis  Distance  from  each  domain  point  to  the  covariance 
matrix. 

8.  Assign  each  point  the  closest  cluster. 

9.  Go  to  step  3  until  converged  technology  driven,  warfighter  focused. 
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